System, method, and computer program product for model-based data analysis

ABSTRACT

A system, method, and computer program product for a model-based data analysis system is disclosed. The method includes the steps of receiving information from one or more respondents that includes at least one response to a question included in a first survey, updating a model based on the received information, and generating a second survey based on the updated model. The method may be implemented by a server application communicating with a client application via a network.

RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application No.62/104,647, filed Jan. 16, 2015, which is herein incorporated byreference.

FIELD OF THE INVENTION

The present invention relates to data analysis systems, and moreparticularly to automated, analysis of datasets collected based onmarket research surveys.

BACKGROUND

Market research surveys are a useful tool for enabling companies totarget particular audiences that are more apt to purchase their servicesor products. For example, market research surveys may be used to findthat male respondents are twice as likely as female respondents topurchase a particular product. Based on the knowledge gleaned from thesurvey results, the company may decide to direct more advertisingdollars to areas that are more heavily associated with a male audience.For example, advertisements may be directed to sporting events or videogames that are conventionally consumed by a heavily male dominatedaudience.

Market research surveys are conventionally conducted through variousmediums such as in-person, over the phone, and, more recently, throughonline systems. Online systems for conducting market research tend to becheaper to implement because they do not require a researcher to ask thequestions and record a respondent's answers, like in-person or over thephone techniques require. However, such systems still require a certainlevel of management by a researcher to create the surveys and analyzethe answers provided by the respondents. For example, a researcher maydesign a set of questions to include in a survey. The survey may then beanswered by 100 respondents. The researcher may then analyze theresponses provided by the group of respondents and, based on theanalysis, adjust the questions included in the survey. The modifiedsurvey may then be answered by additional respondents. The researchermay then analyze the responses provided by both the initial group ofrespondents and the new group of respondents.

As described above, conventional systems for creating market surveys andcollecting data associated with the responses provided by a group ofrespondents tend to rely heavily on manual manipulation of the surveyquestions and survey design. This requires a high level of control by aresearcher conducting the market research and may take a long time toachieve the desired results. For example, each cycle of creating a newsurvey, conducting the survey, and analyzing the results may take asignificant amount of time. Furthermore, in order to arrive atstatistically relevant data, multiple cycles may need to be performed inorder to achieve reliable results. This may stretch the time required toconduct accurate market research into many weeks or months. Thus, thereis a need for addressing this issue and/or other issues associated withthe prior art.

SUMMARY

A system, method, and computer program product for a model-based dataanalysis system is disclosed. The method includes the steps of receivinginformation from one or more respondents that includes at least oneresponse to a question included in a first survey, updating a modelbased on the received information, and generating a second survey basedon the updated model. The method may be implemented by a serverapplication communicating with a client application via a network.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart of a method for dynamically updating a model forgenerating a market research survey, in accordance with one embodiment;

FIG. 2 illustrates a system for managing market research projects,according to one embodiment;

FIG. 3 illustrates the operation of the survey engine of FIG. 2, inaccordance with one embodiment;

FIG. 4 illustrates the operation of the model engine of FIG. 2, inaccordance with one embodiment;

FIG. 5 illustrates the operation of the analysis engine of FIG. 2, inaccordance with one embodiment;

FIG. 6 illustrates a presentation module, in accordance with oneembodiment;

FIGS. 7A-7E illustrate a customer portal used by a market researchanalyst to access the server application, in accordance with oneembodiment;

FIG. 8A illustrates a flow chart of a method for generating a surveybased on one or more models, in accordance with one embodiment;

FIG. 8B illustrates a flow chart of a method dynamically generating asurvey, in accordance with one embodiment; and

FIG. 9 illustrates an exemplary system in which the various architectureand/or functionality of the various previous embodiments may beimplemented.

DETAILED DESCRIPTION

Market research surveys are a useful tool for collecting large amountsof data about a certain population. The surveys comprise a set ofquestions that are designed to provide details about the population. Aset of respondents is chosen from the population to provide answers tothe set of questions. Each respondent may answer the set of questionsbased on their personal traits, qualities/preferences, and/orexperiences. The resulting dataset collected from the set of respondentsmay then be analyzed to provide insight into the market associated withthe population.

The creation and availability of online tools and systems for conductingmarket research may make it easier for marketing personnel to create andconduct surveys. Furthermore, the content of the surveys may be updateddynamically in response to the answers provided by a specific respondentand/or the answers provided by a number of previous respondents.Modeling may be performed to increase the effectiveness of a particularsurvey. In one embodiment, a model anticipates the responses provided bya sample of a given population and changes the questions in a surveybased on an analysis of the previous responses received from one or morerespondents. For example, a model may predict that 45% of men regularlypurchase tickets for professional sporting events while 75% of women donot regularly purchase tickets for professional sporting events, basedon the responses of previous respondents to other surveys. A survey maybe designed that includes a first question that asks the respondentwhether the respondent's gender is male or female. If the respondentindicates their gender is male, then the survey content may be updatedto include more questions related to sporting events. However, if therespondent indicates their gender is female, then the survey content maybe updated to ask different questions unrelated to sporting events.

In addition to the responses provided by the respondents to surveyquestions the model may also be based on additional data. In oneembodiment, the model may be based, at least in part, on behavioral datacollected outside of the surveys. For example, the behavioralinformation may include data about the types of people that make in-apppurchases or the types of people that gamble at least $2,000 a year inLas Vegas. Such behavioral data can be collected, for example, by miningbusiness records of one or more companies about their customers. Suchdata may be useful in predicting the responses provided by respondentsto specific questions and may be used to tailor the types of questionsincluded in surveys provided to particular respondents. In addition tobehavioral information, other types of information, such as informationabout locations, may be used to affect the model.

In one embodiment, the behavioral data and/or other types of informationmay include data retrieved from a mobile device. For example, arespondent may access an application on a mobile phone of the respondentto complete one or more surveys. However, in addition to the explicitresponse data provided by the respondent to the questions in the survey,the application may also retrieve additional information about therespondent. The additional information may include behavioral data suchas how often the respondent opens the application, what websites therespondent views on the mobile device, whether the respondent listens tomusic on the mobile device, etc. In one embodiment, the additionalinformation may also include contextual information about the mobiledevice, such as a type of device, a location of the mobile device, atime, connections to nearby devices, detection of nearby friends/users,etc. Such additional information may also be utilized by a model to makepredictions. In one embodiment, any contextual information associatedwith the mobile device may be used to make predictions and/or beleveraged in any manner in relation to the survey.

The models may be utilized by a data analysis engine to create dynamicsurveys that are provided to a number of respondents. The dynamicsurveys enable new questions to be asked that tailor the survey to thepopulation responding to the survey. The dynamic nature of the marketresearch surveys described herein enable more useful information to becollected in a shorter timeframe than conventional systems that requirea researcher to manually analyze and change the content of the surveysto get the same results. Furthermore, the models described herein mayalso allow for the surveys to be completed by a fewer number ofrespondents to get statistically relevant results when the receivedresponses correlate with the expected responses predicted by the model.

FIG. 1 is a flow chart of a method 100 for dynamically updating a modelfor generating a market research survey, in accordance with oneembodiment. As shown in FIG. 1, at step 102, a survey engine receivesinformation from one or more respondents. In one embodiment, theinformation may comprise a set of answers to a corresponding set ofquestions included in a survey completed by each respondent. Of course,in one embodiment, each respondent may be included in the one or morerespondents. Each survey completed by a respondent may be the same ordifferent from the surveys completed by the other respondents. In otherwords, each respondent may independently submit answers to a specificsubset of questions included in the survey for that respondent. Theinformation received by the survey engine may include the answers toeach question submitted to the respondent and not every respondent willhave submitted an answer to each question.

As used herein a survey includes a set of questions and a correspondingset of potential answers. For example, each question in a survey mayinclude a plurality of potential answers that a respondent may select asthe respondent's answer to that particular question. In one embodiment,the answers may be provided in a multiple choice format that allows arespondent to select one or more answers as the respondent's answer to aparticular question. In another embodiment, the answers may include afield that can be filled out by the respondent to provide any answer ofthe respondent's choice to a particular question.

In one embodiment, a market research analyst may design a set of surveysassociated with a particular market research project. The marketresearch project may include more than one survey that asks questions todifferent sets of respondents to generate data about a population (i.e.,the market participants), a sample of which comprises the respondents.By including different surveys in the market research project, changingthe order of questions in the surveys, or including related, butslightly different, questions in each of the different surveys in themarket research project, the market research analyst may glean moreaccurate assumptions about the population than if a single survey wereasked to all of the respondents sampled from the population. Forexample, results from long surveys may be less accurate than resultsfrom short surveys because the respondents become less enthusiasticabout providing accurate answers to long surveys. By splitting a largenumber of questions into different smaller surveys completed bydifferent subsets of the respondents, the results may be more accurate.

At step 104, an analysis engine updates a model based on the informationreceived from the one or more respondents. The model may be used topredict responses from one or more respondents based on responsesprovided by other respondents, in the same or different market researchproject as well as any other data used by the model. The analysis enginemay receive a data structure that includes model parameters needed forthe analysis engine to make a prediction for a particular questionincluded in a survey completed by one or more respondents. The analysisengine may update the model based on the information received from thesurvey engine and then execute instructions that implement a specificalgorithm to analyze the information and create an output. The outputmay include an accuracy of the prediction calculated by the analysisengine. After the model has been run, one or more actions may be takento update the surveys provided to the one or more respondents.

For example, a model may predict that a specific percentage of womenenjoy shopping for clothes while a smaller percentage of men enjoyshopping for clothes. Another model may also predict that a particularpercentage of the market participants are men versus women. Yet anothermodel may also predict that, of the women that enjoy shopping forclothes, a percentage of those women tend to spend more than $300 whileshopping at a particular store. In general, for each potential questionincluded in the survey, the model(s) may be run by the analysis engineto predict what percentage of respondents will select each answercorresponding to that particular question as well as provide anestimation of the accuracy of that prediction.

The model(s) may also correlate different questions that may be includedin a particular survey such that an answer to a particular question willbe more statistically relevant to a given population of respondents thathas provided a specific answer to a previously provided question in thesurvey. In other words, the model(s) may identify relationships betweenrelated and/or unrelated questions. For example, a question aboutpurchasing tools from a hardware store may be more statisticallyrelevant if a respondent answers that the respondent is male than if therespondent is female. Similarly, a question about purchasing makeup froma beauty store may be more statistically relevant if a respondentanswers that the respondent is female than if the respondent is male.Although the above example relates an immutable characteristic of therespondent to the statistical relevance of a question, behavioral traitsor other traits assumed based on answers to previously providedquestions may also be used to determine statistical relevance of anygiven question. For example, the model may identify that people thatlike dogs are more likely to shop at Target than people that like cats.Even though the questions appear unrelated, the model may discover acorrelation between the questions based on an analysis of the responsedata.

In one embodiment, a model may not exist when a market research projectis initiated. For example, a model may not be effective without somethreshold amount of survey response data from a number of respondents.In such an embodiment, the model is not run by the analysis engine untila threshold number of respondents have completed a survey. Once thethreshold number of respondents have completed the survey, the analysisengine may begin to run the model and update the surveys provided to therespondents.

At step 106, the survey engine updates one or more surveys based on theoutput of the model. In one embodiment, the output of the analysisengine may cause one or more surveys to be removed from an active surveylist (i.e., deactivated), which indicates which surveys are available tobe completed by the respondents. In addition, one or more additionalsurveys may be added to the active survey list (i.e., activated). One ormore respondents may then be able to complete the new surveys added tothe active survey list. The surveys may be pre-defined by a marketresearch analyst. The particular surveys that are activated ordeactivated may be specified based on the output of the model and,therefore, the progression of the market research project (i.e., theorder and number of questions asked to the respondents) is dependent onthe model run by the analysis engine.

In another embodiment, the survey engine may automatically select a setof questions from a pool of available questions to be included in asurvey based on the output of the analysis engine. In one embodiment,one or more models may be updated after each respondent completes asurvey. The model may be updated based on the answers provided by therespondent to questions in the survey completed by the respondent. Asmore and more respondents complete surveys, the model(s) associated withthe questions in the surveys may more accurately predict the types ofresponses that are likely to be provided by a particular respondentselected from the pool of market participants. New surveys may begenerated based on the model(s) as more and more respondents provideanswers to the surveys transmitted to the respondents. The new surveysmay include different questions from the earlier surveys to collectdifferent data about different subjects of interest to the marketresearch analyst. In other words, the surveys may be generateddynamically based on the execution of the model, which is updated basedon responses provided by the respondents.

For example, at the start of a market research project, a model maypredict that half of the market participants are male and half of themarket participants are female. A survey may be generated that is sentto a thousand respondents selected from the pool of market participants.Of the thousand respondents sent the survey, 250 responses are receivedthat include answers to the questions included in the survey. The modelor models may be updated to reflect that of the 250 responses received,150 respondents indicated that they were women while 100 respondentsindicated that they were men, indicating that the previous assumption ofthe male/female ratio within the pool of market participants may bewrong. Similarly, the model(s) may also be updated to reflect the otheranswers provided by the respondents and/or determine whether there isany statistical correlation between two or more questions/answers. Theupdated model(s) may then be used to generate one or more new surveyswith different questions or a different order to the questions. Forexample, because the population of the market participants appears toskew more towards females than males, the survey questions may bechanged to ask questions that are more statistically relevant to afemale audience (e.g., questions related to shopping for clothes versusquestions related to shopping for tools, etc.).

In one embodiment, model(s) and generated survey(s) may be used to allowa market research analyst to test a hypothesis. For example, in oneembodiment, market research analysts may expect buyers of a certainproduct to be mostly male. As data is collected, a score (e.g.likelihood score, etc.) may be presented to the market researchanalyst(s), indicating an initial result of whether their expectation(s)was/were correct. In some embodiments, a threshold of data (e.g. aminimum of 25 individuals, etc.) must be collected before a hypothesismay be tested. In other embodiments, immediate unfiltered feedback maybe presented indicating initial results of testing a hypothesis.Further, in one embodiment, if the results of testing a hypothesis arecorrect (i.e. the data collected correlates with expected results,etc.), the survey subject to the hypothesis testing may be used forgeneral use (e.g. sent out to additional individuals, etc.).

In another embodiment, the set of questions included in the survey maybe generated dynamically based on the model. For example, a firstquestion in the survey may be selected to ask a respondent whether therespondent is male or female. The survey engine will then select asecond question to include in the survey based on the output from themodel. For example, if the respondent indicates that the respondent isfemale, then the survey engine will select a statistically relevantquestion for females, whereas if the respondent indicates that therespondent is male, then the survey engine will select a statisticallyrelevant question for males. As the respondent continues to answerquestions of the survey, the survey engine will continuously selectadditional questions for the survey, based on the responses previouslyprovided within that survey as well as responses previously provided byzero or more other respondents.

More illustrative information will now be set forth regarding variousoptional architectures and features within which the foregoing frameworkmay or may not be implemented, per the desires of the user. It should bestrongly noted that the following information is set forth forillustrative purposes and should not be construed as limiting in anymanner. Any of the following features may be optionally incorporatedwith or without the exclusion of other features described.

FIG. 2 illustrates a system 200 for managing market research projects,according to one embodiment. The system 200 includes a networkenvironment including software configured in a client-serverrelationship. The network environment includes a server computer 202connected to a network 250. In addition, one or more client computers204 are connected to the server computer 202 via the network 250. In oneembodiment, the network 250 includes a number of devices configured tocommunicate via the standard Internet Protocol Suite (TCP/IP). Thenetwork 250 may include a number of routers, bridges, switches, etc.connected via a number of different physical interfaces such as a fiberoptic interface (e.g., SONET, etc.), a wired interface (e.g., Ethernet,etc.), or a wireless interface (e.g., Wi-Fi, Bluetooth, etc.). In oneembodiment, the network 250 is the Internet. In another embodiment, thenetwork 250 may be a private network, local area network (LAN), or thelike.

The server computer 202 may include one or more processors, a volatilememory such as SDRAM, a non-volatile memory such as a hard disk drive(HDD), a solid state drive (e.g., Flash memory), an optical drive (e.g.,CD-ROM, DVD-ROM, etc.), a network interface controller (NIC), a bus orother communications network, one or more input devices (e.g., akeyboard, a mouse, etc.), and/or one or more output devices (e.g., adisplay device, speakers, etc.). The server computer 202 may alsoinclude software, stored in the non-volatile memory and loaded into thevolatile memory, such as an operating system and one or moreapplications. The operating system and/or one or more applications maybe executed by the one or more processors of the server computer 202.

As shown in FIG. 2, the operating system, when executed by a processor,generates an operating environment 210 within which the one or moreapplications may be executed. The operating environment 210 may includethe processes associated with each active application being executed bythe processor, memory (virtual or physical) that is allocated to eachprocess, a set of functions provided by the operating system, and soforth. The operating environment 210 includes a server application 215,which may be configured to provide a service for managing marketresearch projects. As used herein, the service may be accessible via thenetwork 250 by the one or more client computers 204.

In one embodiment, the server application 215 may include a number ofsoftware modules or engines that provide a specific functionality of theservice. As shown in FIG. 2, the server application 215 includes asurvey engine 232, an analysis engine 236, and a model engine 234.Although not shown explicitly, the server application 215 may alsoinclude an interface module that enables market analysts to access thefunctionality of the server application 215 via a client over thenetwork 250 and a presentation module that generates visualrepresentations of the survey data provided by respondents. Of course,any other modules associated with the server application and used forproviding additional features of the analysis may be included as needed.

The survey engine 232 may access a survey database 242 that includessurveys, a pool of available questions and corresponding answers,response data provided by respondents, and the like. The model engine234 may access a model database 244 that includes information associatedwith modeling the market and representing correlations between surveyquestions and/or responses. The survey database 242 and the modeldatabase 244 may be stored in a memory associated with the servercomputer 202. In one embodiment, the memory is a local, non-volatilememory such as a HDD or an array of HDDs. In another embodiment, thememory is a network-accessible memory such as a storage as a service(SaaS) like that provided by the Amazon® S3 web services solution. Itwill be appreciated that in either case, at least a portion of thedatabase may be fetched into a local volatile memory such as SDRAMconnected to the processor of the server computer 202 so the portion maybe accessed by the survey engine 232 and/or model engine 234.

As also shown in FIG. 2, the analysis engine 236 may access the surveydatabase 242 and the model database 244. For example, the analysisengine 236 may access questions in the survey database 242 to retrieveresponses to completed surveys provided by one or more respondents. Theanalysis engine 236 may also access model data included in the modeldatabase 244 to update the model based on responses received from one ormore respondents.

Each of the client computers 204 may include a central processing unit(CPU), a graphics processing unit (GPU), a volatile memory, anon-volatile memory, a NIC, one or more input devices, and/or one ormore output devices. The client computers 204 may also include softwaresuch as an operating system and one or more applications. Similar to theserver computer 202, the operating system, when executed by a processor,generates an operating environment 220 within which the one or moreapplications may be executed.

As shown in FIG. 2, in one embodiment, one of the applications mayinclude a browser application 224 for accessing and viewing informationaccessible over the network 250. For example, the browser applicationmay be configured to transmit HTTP requests to the server application215 executed by the server computer 202. The server application 215 maytransmit HTTP responses to the browser application 224 on the clientcomputer 204. The HTTP responses may include markup language documentssuch as HTML documents or XML documents that may be formatted to bedisplayed in a window of the browser application 224. The browserapplication 224 may be configured as a thin client, such as a portalaccessible by a web browser, to be used to access the functionality ofthe server application 215 remotely. In other words, the portalcomprises one or more markup language documents that are served to theclient computer 204 and displayed in the browser application 224 in awindow on the display device of the client computer 204. The markuplanguage documents may include an interface, such as a form, thatenables a user to input information and call the functions of the serverapplication executed on the server computer 202 from the browserapplication 224 of the client computer 204.

In another embodiment, one of the applications may include a clientapplication 222. The client application 222 may be embodied in astandard binary executable that is executed within the operatingenvironment 220 provided by the operating system of the client computer204. The client application 222 may include a graphical user interface(GUI) that includes a number of input elements (e.g., text boxes,buttons, menu items, etc.) and/or output elements (e.g., images, text,etc.), referred to herein as user interface (UI) elements. The UIelements may be configured to enable a user to input information andcall the functions of the server application executed on the servercomputer 202 from the client application 222.

The client application 222 or the browser application 224 (hereinafter,generally referred to as the “client”) may be configured to callfunctions of the server application 215 via an application programminginterface (API). The API may define function calls that enable a marketanalyst, via the client, to access functions provided by the serverapplication 215, such as creating survey questions, generating newsurveys, managing a market research project, viewing the data associatedwith responses provided by a number of respondents, and so forth.

FIG. 3 illustrates the operation of the survey engine 232 of FIG. 2, inaccordance with one embodiment. The survey engine 232 enables a datasetto be created, stored, and managed by a market research analyst as wellas facilitating completion of surveys by respondents. A dataset may beassociated with a market research project and includes all informationabout the survey questions asked to respondents, the answers to thequestions submitted by respondents, and demographics data about therespondents. In one embodiment, the dataset may include (1) QuestionFamily data; (2) Question data; (3) Respondent data; (4) Filter data;and/or (5) Survey Response data. Of course, in other embodiments, anytype of data may be associated with a dataset.

The dataset may be stored in the survey database 242. In one embodiment,each of the different types of data may be stored in a different tableof a relational database. For example, question family data may bestored in a first table of the relational database, question data may bestored in a second table of the relational database, respondent data maybe stored in a third table of the relational database, filter data maybe stored in a fourth table of the relational database, and surveyresponse data may be stored in a fifth table of the relational database.In another embodiment, the dataset may be stored in one or moredatabases. For example, the question family data and question data maybe stored in one database, the respondent data and filter data may bestored in another database, and the survey response data may be storedin yet another database.

A “Question Family”, as the term is used herein, may refer to a set ofrelated questions. An entry in the dataset may include an identifier forthe question family, a descriptive name for a question family, and atype of the question family. In one embodiment, the valid types ofquestions may be: (1) Answer Questions; (2) Descriptive Questions; and(3) Weight Questions. Descriptive questions may include questions thatdescribe demographic data about the respondent, such as the respondent'sgender, age, or location. Weight questions may include questions thatrequest an answer to the question in the form of a numeric response andcan be used to weigh a respondent's answers. For example, a question mayask a respondent to specify whether they like or dislike a particularproduct on a scale of 1 to 5. Answer questions may include all questionsthat do not fall into the descriptive or weight categories. For example,an answer question can be phrased as “Have you purchased a lawn mower inthe last 6 months: Yes or No?” The specific question types describedabove are provided as illustrative and should not be construed aslimiting in any way. Other classifications of question types arecontemplated as being within the scope of the present disclosure. Insome embodiments, the types of questions included in the survey may bedefined by the market research analyst.

A “Question”, as the term is used herein, may refer to a definition of aquestion to be included in a survey. An entry in the dataset may includean identifier for the question, the text associated with the question(i.e., the prompt to be displayed on screen to the respondent while therespondent is participating in the survey), an identifier of thequestion family the question is included in (if any), and a set of validanswers to the question. For example, a question may be defined as Qn_1,associated with the text of “What is your ethnicity?”, included in theQF_1 question family, and have valid answers specified as:“Caucasian/White”; “Hispanic/Latino”; “Asian/Pacific Islander”;“African-American/Black”; and “Other”.

Questions may have answers ranging from a selection of a set of validanswers, an unbounded numeric response, or a numeric response limited toa specific scale. In some embodiments, questions may also allow anunbounded answer to be entered by a respondent. For example, arespondent could type an answer into a text box. Such answers may have awide range of possible answers given the text entered by the user is notlimited to a specific set of choices. Examples of the types of answersprovided are: a number from 1 to 99 in response to the question “What isyour age?”; a number from 1 to 10 in response to the question “On ascale of 1 to 10, how much do you enjoy the taste of pepperoni on yourpizza?”; and a selection of an ethnicity type from a list such as thelist set forth above. In one embodiment, the valid answers may beprovided to the respondent in a multiple choice format and therespondent is prompted to choose the correct answer from the list ofavailable valid answers. In another embodiment, the valid answers mayspecify a test to determine whether an answer provided by the respondentis valid. For example, valid answers may be specified as any numberbetween 1 and 100. Any answer provided by the respondent will then bechecked to make sure the answer is a number between 1 and 100.

A “Respondent”, as the term is used herein, may be an individual thatcompletes a survey within a population targeted by the market researchproject. In other words, a set of respondents represent a sample of thepopulation. The dataset may include an entry for each respondent thathas previously opted-in to participate in a market research project. Theentry in the dataset may include a name of the respondent, a gender ofthe respondent, contact information for the respondent, ausername/password for the respondent to login to a portal for completingsurveys, and so forth. Additionally, information associated with arespondent may include a unique identifier which may persist across twoor more surveys and/or databases. In one embodiment, such an identifiermay allow a market research analyst to track an individual's responsesacross a range of different surveys and data sources.

A “Filter”, as the term is used herein, may refer to a filter that isapplied to select a subset of data from the dataset. For example, afilter may be defined that selects all respondent data associated withMale respondents or all respondent data associated with respondents aged19-35. Such filters may be defined in the dataset as simple text in anyvalid format. For example, a filter may be defined as a SQL (structuredquery language) query of the dataset included in the survey database242. Filters may be combined by a particular analyst to reduce theamount of data being analyzed after one or more surveys have beencompleted by a set of respondents.

In one embodiment, the dataset may also include survey response data.Survey response data may correspond to the answers selected by therespondents to the questions included in a survey. Each entry associatedwith the survey response data may include an identifier of the questionthat was answered, an identifier of the respondent that answered thequestion, and the respondent's provided answer to the question.

In one embodiment, the survey engine 232 enables the dataset to becreated by the market research analyst. When a market research analystwants to start a market research project, the market research analystmay access the server application 215 via the client. The market analystmay then create at least a portion of the survey database 242 for themarket research project by entering data into the client. For example,the market research analyst may populate the different tables in thesurvey database 242, such as a question family table 311, a questiontable 312, a respondent table 313, and a filter table 314, with theappropriate entries.

The survey engine 232 includes a number of different components. In oneembodiment, the survey engine 232 includes a survey controller 320, adatabase manager 321, and a survey generator 322. The survey controller320 facilitates communication with the client. The survey controller 320receives requests from the client to, e.g., create a new market researchproject. The survey controller 320 may process the request and transmita request to the database manager 321 to create a new database in amemory associated with the server computer 202. The database manager 321may create the database, such as the survey database 242, in the memory.The market research analyst, via the client, may then add data to thedatabase by entering information into fields in the client andtransmitting the information to the survey controller 320 and thedatabase manager 321 to add an entry to one of the tables in thedatabase. For example, the market research analyst may utilize a formdisplayed in the client to enter information related to questions thatthe market research analyst would like to include in the surveys. Whenthe information has been entered the market research analyst submits theform on the client, which is then transmitted to the survey controller320 via an HTTP request. The survey controller 320 parses the HTTPrequest and transmits a request to the database manager to add an entryto the question table 312 in the database.

In one embodiment, the server application 215 may also access additionaldata stores to import information into the newly created database. Forexample, the server application 215 may have access to a list ofrespondents that have previously opted-in to participate in marketresearch. The market research analyst may select a subset of respondentsfrom the list of respondents to import into the respondents table 313 ofthe survey database 242. In one embodiment, the market research analystmay apply a filter to the list of respondents to select the respondentsto include in the respondents table 313 of the survey database 242, suchas selecting all respondents in a particular state.

The server application 215 may also have access to a question dictionarythat includes a listing of questions used in other market researchprojects. Market research analysts that have created questions as partof other market research projects may save questions they have createdwithin their particular market research project to the questiondictionary to be accessed and shared with other market researchanalysts. These standard questions may enable faster creation of surveysby allowing a market research analyst to quickly select a group ofquestions rather than requiring all questions to be re-entered. In oneembodiment, the server application 215 may access both a standardquestion dictionary shared by all market research analysts and a privatequestion dictionary that can only be accessed by that particular marketresearch analyst. In other words, the private question dictionaryincludes questions previously saved by that market research analyst as apart of other market research projects, but not shared with other marketresearch analysts.

Once the market research analyst has populated the survey database 242,the market research analyst may create one or more surveys 350. Eachsurvey 350 includes a plurality of questions stored in the questiontable 312 of the survey database 242. In one embodiment, the surveys 350are manually created by the market research analyst by selecting a setof questions to include in the survey 350 and determining an order forthe questions in the survey. In another embodiment, as described in moredetail below, the surveys 350 may be automatically created based on amodel.

The market research analyst may create a survey 350 by selecting thequestions to include in the survey 350 using a form displayed by theclient. The market research analyst may add each question to a list in aspecific order and then submit the ordered list of questions to thesurvey engine 232 via an HTTP request. The survey controller 320 mayparse the HTTP request and transmit a request to the survey generator322 to create a new survey. The survey generator 322 may then create asurvey 350 and store the survey 350 in a memory associated with theserver computer 202. In one embodiment, the survey 350 includes anordered list of pointers to the questions in the survey database 242.The survey 350 may also include metadata such as a list of respondentsthat the survey 350 is distributed to, an author of the survey 350, andan identifier to uniquely identify the survey 350.

In one embodiment, the survey 350 may include questions that areconditionally included in the survey 350 based on responses provided bythe respondent. In other words, the contents of the survey 350 may bedynamic based on the responses provided by the respondent. In onetechnique, each question in the survey may be optionally associated witha condition(s) that determines whether a question in the survey 350 willbe displayed to a respondent. For example, a first question may askwhether the respondent is male or female. Based on the respondent'sanswer to the first question, a second question will optionally bedisplayed. For example, a question based on how many hours a week therespondent watches live sporting events may only be displayed if therespondent answered the first question with “male”. The conditions maybe specified as a Boolean test that compares the response provided bythe respondent to a specific question to a target response. If aquestion is not displayed to the respondent, then an answer to thatquestion will not be included in the response data submitted by therespondent when they have completed the survey.

The survey controller 320 also manages the administration of the surveys350 to the respondents. In one embodiment, the market research analystmay specify a subset of respondents to whom the survey 350 should besubmitted. The market research analyst may select each of therespondents individually from a list of all respondents. Alternatively,the market research analyst may select the respondents by applying afilter to the list of all respondents. Finally, the market researchanalyst may specify a total number of respondents to submit the surveyto and the survey controller 350 may randomly select the total number ofrespondents from the list of all respondents. Once the subset ofrespondents has been selected, the survey controller 320 may facilitatenotification to each of the respondents in the subset of respondentsthat the survey 350 is available. The survey controller 320 may cause amessage, such as an email, a text message, or a recorded voice message,to be sent to each respondent at either the respondent's email addressor phone number. In another embodiment, the message may be associatedwith a portal accessible by the respondent on the web. When therespondent logs into the portal, the message may be accessed. Forexample, the portal may include an electronic mail aspect that includesthe message. Alternatively, the portal may simply display the messagewithin one of the elements of the portal, such as text within a <p>element at the top of a markup language document.

In one embodiment, the message may include a link, such as a URL(uniform resource locator), that the respondent can click to open aninterface in a browser application 224 in order to complete the survey350. The interface may be associated with a portal or web page(s) thatenable the respondent(s) to complete surveys 350. The browserapplication 224 generates an HTTP request associated with the link thatis transmitted to the survey controller 320. The survey controller 320may return a markup language document to the browser application 224that includes an interface that enables the respondent to enter therespondent's identification information. The identification informationmay be a username and password previously associated with a particularrespondent. The respondent may enter their username and password in theinterface and submit the information to a specific URL in an HTTPrequest transmitted back to the survey controller 320. The URL may havea specific route that is associated with a specific market researchproject. The survey controller 320 may authenticate the respondent usingthe provided username and password. Once the survey controller 320 hasauthenticated the respondent, the survey controller 320 may check anactive survey list associated with the market research project. If thereare no available surveys in the active survey list, then the surveycontroller 320 may return an HTTP response to the browser application224 that includes a message indicating there are no surveys available tobe completed by the respondent. However, if at least one survey isincluded in the active survey list, then the survey controller 320 mayreturn an HTTP response to the browser application 224 that includes aninterface for completing a survey selected from the active survey list.

The interface may include a form that displays at least one questionincluded in the selected survey and at least one element enabling therespondent to submit an answer to the question. In one embodiment, theHTTP response includes a markup language document that includes aquestion and, optionally, a list of one or more potential answers thatare displayed as text that is visible to the respondent and a form forsubmitting an answer to the question. When the respondent submits theform with the respondent's answer to the question, the browserapplication 224 generates a new HTTP request that includes the form dataand transmits the HTTP request to the survey controller 320. The surveycontroller 320 parses the HTTP request to identify the respondent'sanswer to the question. The respondent's answer may be checked to makesure the answer is a valid answer. If the answer is a valid answer, thenthe survey controller 320 transmits the answer to the database manager321 to store the answer in the survey response table of the surveydatabase 242 (or in a separate database for storing survey responsedata). The survey controller 320 may then transmit an HTTP response thatincludes a markup language document including a next question in thesurvey 350. This process is repeated until all questions in the survey350 have been completed by the respondent. Once the survey 350 iscomplete, the survey controller 320 may transmit an HTTP response to theclient that includes a message thanking the respondent for completingthe survey 350. Although the HTTP responses described above only includea single question, multiple questions may be included in each markuplanguage document such that each HTTP request transmitted by the clientincludes answers to more than one question answered by the respondent,thereby requiring fewer messages to be transmitted between the clientand server to complete the survey 350.

In one embodiment, the survey controller 320 may be configured to enablea respondent to partially complete a survey 350, log out of the portal,and login to the portal at a later time to complete the survey 350. Thesurvey controller 320 may store a pointer to a current question in thesurvey 350 that indicates a location (i.e., question number) that therespondent has reached in the survey 350. When the respondent enterstheir login information in the portal, the survey controller 320 maycheck to see if there are any partially completed surveys 350 in theactive survey list. If there is at least one partially completed survey350, then the survey controller 320 may prompt the respondent tocomplete the survey 350. If the respondent wishes to complete the survey350, then the survey controller 320 may determine the next question tobe displayed to the respondent based on the pointer. Administration ofthe survey 350 will then continue as if the respondent had never stoppedtaking the survey 350. Additional information may also be stored when arespondent partially completes a survey 350, such as a total number ofquestions already completed, a total time elapsed since the survey 350was started, etc., and such information may be utilized when completingthe survey 350.

In this manner, the survey controller 320 may help facilitateadministration of the surveys 350 to one or more respondents. As moreand more respondents complete the survey 350, the market researchanalyst will be able to analyze the data. The market research analystmay also create additional surveys 350 as part of the market researchproject in order to gain more insight into the market. In addition, aswill be described in more detail below, after each survey 350 iscompleted by a respondent, the survey controller 320 may transmit arequest to the analysis engine 236 in order to analyze the respondent'sanswers to the survey 350 and update one or more models associated withthe market research project.

In another embodiment, the survey controller 320 provides an interfacefor receiving response data generated by a third party application. Forexample, the survey controller 320 may generate surveys 350 that areexported to the third party application. The third party application mayfacilitate the administration of the survey to the respondent(s). Theresponse data received from the respondent(s) may then be transmittedfrom the third party application to the survey controller 320. In oneembodiment, the third party application may be a custom applicationinstalled on a computing device, such as a cell-phone of a respondent.The custom application may enable the respondent to easily complete thesurvey via their computing device. Of course, in other embodiments, thesurvey may be integrated into any third-party application, and/ormodified such that one or more features associated with the survey maybe accessed via a third-party application.

In one embodiment, resources associated with the model and/or surveysystem may be provided via an API. In such an embodiment, a third partyapplication and/or resource and/or service may have a separate platformsystem, but may use the API to leverage one or more features associatedwith the model and/or survey system. In this manner, the model and/orsurvey system may function as a separate product to be integrated intoother third party platforms and/or systems. Further, in one embodiment,any resource (e.g. engine, controller, etc.) associated with the modeland/or survey system may be made available via an API. Of course, inother embodiments, any combination of third party resources andresources associated with the model and/or survey system may occur inany manner.

FIG. 4 illustrates the operation of the model engine 234 of FIG. 2, inaccordance with one embodiment. The model engine 234 enables a marketresearch analyst to generate models for predicting answers provided byrespondents to a particular question. In other words, model(s) may berun to predict the probability of a respondent to provide a specificanswer to a particular question within a given population. As describedin more detail below, the model(s) may be run by the analysis engine 236to determine whether a particular question should be included in asurvey by estimating whether an expected answer submitted by aparticular respondent would result in a statistically relevantdatapoint. For example, in one embodiment, the model may determine that90% of respondents that are female and have children shop for groceriesat least one time per week. The model may also predict that the 90%response rate is statistically relevant after at least 50 respondents inthe population have submitted an answer to the question. Thus, based onthe model, a survey may not include a question about how often therespondent shops for groceries if the respondent is female and haschildren after at least 50 respondents have answered said question.Additionally, in one embodiment, if a question is predicted by otherquestions, then it may be removed unless it is used to predict theresponses to the other questions. For example, in one embodiment, if therespondent is female and has children, then the model may return thatthe respondent is very likely to shop for groceries once a week.However, in such an embodiment, the question focused on grocery shopping may only be removed if it isn't used to predict other questions(e.g. the type of grocery produce the individual purchases weekly,etc.). In this manner, therefore, questions presented may be filteredand/or removed if it not used to predict responses to one or more otherquestions.

In one embodiment, the prediction may be generated by calculating arelationship between the target question and a target response withother data associated with the same survey, different surveys, or otherdata. In other words, a relationship may refer to a correlation betweena particular answer to a target question and (1) an answer to anotherquestion by the same respondent in the same survey; (2) an answer toanother question by a different respondent or respondents to the samesurvey or different surveys; or (3) other data that is unrelated toquestions in a survey (e.g., purchase data for a specific store, websiteusage, etc.). The analysis engine 236 may reduce “data ignorance” abouta target question/response by finding such correlations such that eachtarget question and target response is not an isolated data point. Byperforming the analysis repeatedly as new responses are received, theanalysis engine 236 will generate a set of predictions for a targetquestion. In another embodiment, the analysis engine 236 may analyze apredefined model or a predefined set of relationships in order to testthe validity, relevance, or accuracy of the predefined model. Thepredefined model may be analyzed via a set of question specificcoefficients that are used to weight the responses provided to thequestion by one or more respondents in order to calculate the accuracyof the predefined model.

A model may refer to a set of calculations or computations based on thesurvey response data, one or more additional datasets such as behavioraldata, and model parameters specified for the model. In one embodiment,the model parameters may be stored in a data structure that storesinformation needed by the analysis engine 236 to run the model. Eachmodel data structure 450 may be associated with a particular model andeach model may correspond with a particular question included in thesurvey database 242. Once a market research analyst has created a newquestion and added the question to the survey database 242, the marketresearch analyst may be prompted to specify the model parameters for thequestion. Consequently, a plurality of models data structures 450 may beassociated with a particular survey 350 corresponding to a plurality ofmodels to be run by the analysis engine 236. In another embodiment, amodel data structure 450 may be associated with a particular survey 350such that the model data structure 450 contains model parameters relatedto a plurality of the questions included in the survey 350. In yetanother embodiment, a model data structure 450 may be associated with aparticular market research project such that the model data structure450 contains model parameters related to a plurality of questionsincluded in all of the surveys 350 associated with the market researchproject.

In one embodiment, a market research analyst may generate a model datastructure 450 using an interface displayed by the client. The marketresearch analyst may enter the information for the model data structure450 in one or more fields of the interface and submit the information tothe server application 215 via an HTTP request. The model controller 420in the model engine 234 may be configured to parse the HTTP request toretrieve the information for the model data structure 450. Theinformation may be transmitted to a model generator 421 that isconfigured to format the data structure for the model data structure 450to include the information provided by the market research analyst andstore the model data structure 450 in the model database 244. The modelcontroller 420 may transmit an HTTP response to the client that informsthe market research analyst whether the creation of the model datastructure 450 was a success or a failure. The market research analystmay use the interface to create a plurality of model data structures 450corresponding to a plurality of the questions included in the surveys350.

Each model data structure 450 includes a number of parameters thatenable the analysis engine 236 to run a model. In one embodiment, theparameters include a survey identifier, a question identifier, a targetresponse, an accuracy threshold, a computation method, and stoppingcriteria. The survey identifier may be a key that specifies a particularsurvey 350 in the survey database 242. The question identifier may be akey that specifies a particular question in the identified survey 350.The target response identifies a particular response to the question forwhich a prediction is to be calculated. The target response may be anidentifier of one of the valid answers associated with the question.Alternately, the target response may be a range of values that match twoor more valid answers to the question. For example, the target responsecould be a range of 1-3 on a scale of 1-10 or the target response couldbe a range of 18-35 in response to a question related to therespondent's age. The accuracy threshold identifies a minimum level ofaccuracy at which the prediction generated based on the model is assumedto be correct. Typically, the accuracy threshold is related to a numberof respondents that have submitted an answer to the question. Thecomputation method enables the market research analyst to select theparticular algorithm used to make the prediction. The availablealgorithms may be hardcoded into the model engine 234. Alternately, themarket analyst may specify a new algorithm by selecting a file includingsource code implementing the algorithm or selecting a file including abinary executable that implements the algorithm. Finally, the stoppingcriteria may specify one or more actions to be taken when a predictiongenerated by the model meets or exceeds the accuracy threshold.

Although the model data structure 450 may include parameters associatedwith a single question, in other embodiments, the model data structure450 may include parameters associated with a plurality of questions. Forexample, the question identifier could be modified to include an arrayof question identifiers for multiple questions within a survey 350 andthe target response could be modified to include an array of identifiersfor responses to the questions identified within the array of questionidentifiers. In other embodiments, the model data structure 450 mayinclude parameters related to every question in a particular survey 350.In yet other embodiments, the model data structure 450 may includeparameters related to every question associated with a plurality ofsurveys 350 in a market research project. It will be appreciated thatthe list of parameters provided above is illustrative and notexhaustive. In some embodiments, the model data structure 450 may notinclude one or more parameters described above. Furthermore, one or moreadditional parameters may be included in lieu of, or in addition to, theparameters described above.

FIG. 5 illustrates the operation of the analysis engine 236 of FIG. 2,in accordance with one embodiment. The analysis engine 236 is configuredto generate predictions related to each target response to a targetquestion specified by the model data structure 450. The analysis engine236 may access the survey database 242, including survey response datafor one or more surveys 350 provided by one or more respondents, as wellas the model database 244, including one or more model data structures450.

In one embodiment, the survey controller 320 may be configured totransmit a message to the analysis controller 520 whenever a respondentcompletes a survey 350. The message may cause the analysis controller520 to run a model or models related to one or more questions in thesurvey 350. In one embodiment, the analysis engine 236 maintains anactive model list, which is a file that includes a list of model(s) thatshould be executed after a survey is completed. Each model may generatean output related to a question specified by the corresponding modeldata structure 450. A different active model list may be maintained foreach survey 350 included in the market research project. The activemodel list(s) may be managed manually by the market research analyst. Aninterface displayed in the client may be used to modify the active modellist for a given survey 350. By submitting a form via the client, theserver application 215 may store a file including the active model listin a memory associated with the server computer 202.

In response to receiving the message from the survey controller 320, theanalysis controller 520 may read the active model list and configure thecomputation engine 521 to run a model for each model data structure 450included in the active model list. For a particular model data structure450 included in the active model list, the analysis controller 520 mayconfigure the computation engine 521 to execute an algorithm specifiedby the computation method parameter in the corresponding model datastructure 450. The computation engine 521 may then run the model, atleast in part, by executing the algorithm to generate a prediction,based on the response data provided by one or more respondents to thequestion specified by the corresponding model data structure 450. Thecomputation engine 521 may also utilize other data to generate theoutput of the model. In one embodiment, the computation engine 521 isconfigured to serially run models for each model in the active modellist. In another embodiment, the computation engine 521 may beconfigured to run two or more models in parallel.

In yet another embodiment, the analysis controller 520 may generate amodel dynamically. For example, based on previous analysis performed byother models, the analysis controller 520 may change aspects of how amodel works, excluding basic components of the model, adding basiccomponents of the model, or replacing certain components in a model withother components in a model. Even though the components of the model maychange, the computation method used to calculate various variables(e.g., mutual information entropy, correlation measurements, etc.) maystay the same.

In one embodiment, a model may not be accurate until a certain amount ofresponse data for a question has been received and stored in the surveydatabase 242. Thus, the analysis controller 520 may be configured toonly run the model for a question if there is a threshold number ofresponses to a question stored in the survey database 242. In anotherembodiment, a “soft launch” of a survey may be performed where a surveyis sent to a small number of respondents to be completed. The model(s)may not be run when the response data is received from these surveys.Instead, the response data provides an initial set of response data onwhich the models are based, in order for the models to provide moreaccurate results when initially run.

In one embodiment, the analysis controller 520 is configured torecognize predictive patterns in the responses to questions provided byone or more respondents. For each model data structure 450 in the activemodel list, the analysis controller 520 may generate an output file thatincludes the prediction generated by the model. For example, the outputfile may include correlation measurements for each question in a survey350 with one or more other questions, in that survey 350 or othersurveys 350, as well as correlation measurements between that questionand any additional data. The output file may be viewed by the marketresearch analyst using the client. The analysis controller 520 may alsotransmit a message to the survey controller 320 that indicates whetherthe generated prediction met or exceeded the accuracy thresholdparameter specified in the corresponding model data structure 450. Ifthe generated prediction met or exceeded the accuracy thresholdparameter, then the analysis controller 520 may cause the one or moreactions specified by the stopping criteria parameter to be executed. Inone embodiment, the stopping criteria may specify that the surveycontroller 320 should deactivate the current survey 350, activate a newsurvey 350, or remove/replace one or more questions from the currentsurvey 350.

In one embodiment, the analysis engine 236 may receive data from one ormore sources, including responses in a current survey 350, responses toother surveys 350, and or data from additional databases, such as datarelated to purchase transactions. The analysis engine 236 may thenprocess each model, either sequentially or in parallel depending onavailable server-side resources, to analyze the data and generate aprediction for each target question associated with a model datastructure 450 in the active model list. Inputs to the analysis (i.e.,model), in addition to the received data, may be utilized within themodel to generate the prediction and may be included in a correspondingmodel data structure 450, hard-coded into the analysis engine 236, orretrieved by the analysis engine 236 from some other storage locationassociated with the server computer 202. The inputs may include, at aminimum, a target question and target response associated with themodel. The inputs may also include an accuracy threshold parameter, apredefined order for selecting variables to be applied within the model,a set of variables to be excluded from the model, a computation method,etc. As used herein, a variable may refer to a question andcorresponding answer provided by one or more respondents.

For each model, the analysis engine 236 produces a prediction, which mayrefer to a set of mutually exclusive conditions that predict theresponses provided by respondents to the target question. For example,the mutually exclusive conditions may include a condition that if arespondent answers a first question with a first response, then thepredicted response to the target question is likely to be X, at apredicted rate of P. The output of the analysis engine 236 may be a filethat lists all of the mutually exclusive conditions for thecorresponding target question and target response. The output may bedisplayed to the market research analyst via one of several methods. Inone embodiment, the set of mutually exclusive conditions may be listedas a set of conditions and an associated predictive power (i.e.,accuracy) for each condition. In other words, the condition may be a setof responses to one or more questions in the survey and the associatedpredictive power may be the predicted percentage of the time that arespondent will answer the target question with the target responsegiven that the respondent's responses to the other questions meet thecondition.

In another embodiment, the set of mutually exclusive conditions may bedisplayed as a histogram where a first axis of the histogram is a groupdefined by a set of conditions and a second axis of the histogram isdefined as the size of the group. In yet another embodiment, the set ofmutually exclusive conditions may also be displayed according to theimpact of individual questions to the mutual information entropy. In oneembodiment, the mutual information entropy may be defined as ShannonEntropy from Claude E. Shannon's paper titled “A Mathematical Theory ofCommunication”, the entire contents of which is incorporated herein byreference in its entirety. In other words, the display may listparticular questions that have the largest effect on the calculatedmutual information entropy associated with the target question andtarget response. The latter visualization may be easier to interpretthan looking at individual conditions. Of course, it is recognized thatother methods and/or techniques (e.g. in addition to information entropyand/or data ignorance, etc.) may be used to select a variable to predicta response.

In one embodiment, to create the prediction, the analysis engine 236 mayanalyze the received data to find the variable that explains the largestchange in entropy for a given target question. The analysis may beperformed according to a “data ignorance minimization” algorithm. Thealgorithm is a recursive process that identifies chains of conditionsthat are associated with the largest mutual information entropy forpredicting a given response, t, to a target question, Q_T. Thealgorithm, as executed by the analysis engine 236, begins by calculatingthe mutual information entropy for each of the valid questions in thesurvey 350. Valid questions may include any questions in the survey 350that are not explicitly excluded based on parameters specified for themodel by the market research analyst or any questions for which there ismore than one response from all respondents (i.e., questions for whichall respondents answer the same way do not have effects on otherquestions that can be analyzed, and are therefore invalid). Then, theanalysis engine 236 selects the valid question in the survey 350 that isassociated with the largest calculated mutual information entropy. Theselected question, and response, becomes the first question in the chainof conditions. The analysis engine 236 creates a set of n chains ofconditions, where n is the number of different possible responses to theselected question. For example, if Q_1 is the first question in thechain and there are two possible responses to Q_1 (e.g., “yes”=1,“no”=2, etc.), then the analysis engine 236 will create two chains ofconditions, a first chain including condition Q_1=1 and a second chainincluding condition Q_1=2.

Then, each chain of conditions is checked to determine a total number ofrespondents associated with each chain. For a particular chain, theanalysis engine 236 may filter the response data by the conditions inthe chain. For example, for the first chain including the firstcondition Q_1=1, the response data may be filtered to generate a workingset of response data that includes all surveys 350 for which arespondent answered Q_1 with the response 1 (i.e., “yes”). In otherwords, response data for respondents that answered Q_1 with an answerother than 1 are excluded from the working set of response data. Thenthe total number of respondents that answered Q_1 with 1 is determined.If the total number of respondents is greater than or equal to athreshold level, then the chain will continue to be parsed (i.e.,conditions will be added to the chain of conditions and new chains willbe formed). However, if the total number of respondents is less than thethreshold level, then the chain will not be parsed any further (i.e., nomore chains will be spawned from the chain and the chain of conditionswill be added to the output for the model).

For a particular chain of conditions created during a previous iterationof the algorithm, the analysis engine 236 calculates the mutualinformation entropy for each of the valid questions included in theworking set of response data. The analysis engine 236 selects the validquestion in the survey 350 that is associated with the largestcalculated mutual information entropy within the working set of responsedata. Then, the analysis engine 236 creates a set of n new chains ofconditions to replace the current chain, where n is the number ofdifferent possible responses to the selected question. For example, forthe first chain of conditions including condition Q_1=1, if Q_2 is theselected question and there are four possible responses to Q_2 (e.g.,“White/Caucasian”=1, “Asian/Pacific Islander”=2, “Black/AfricanAmerican”=3, “Latino/Hispanic”=4, etc.), then the analysis engine 236will create four chains of conditions, a first chain includingconditions Q_1=1 and Q_2=1, a second chain including conditions Q_1=1and Q_2=2, a third chain including conditions Q_1=1 and Q_2=3, and afourth chain including conditions Q_1=1 and Q_2=4. These new chains willreplace the current chain being processed in a set of valid chains.

The process is repeated for each chain produced during the previousiteration of the algorithm that is associated with a total number ofrespondents that is greater than or equal to the threshold level. Onceall new chains have been created during the current iteration of thealgorithm, the total number of respondents associated with each chain ofconditions may be checked, and the process may be repeated for anychains that are associated with a total number of respondents that isgreater than or equal to the threshold level. Once a sufficient numberof iterations has been performed such that all valid chains ofconditions are associated with a total number of respondents is lessthan the threshold level, then the output may be produced that includesall of the valid chains. Alternatively, the parsing of the chains may beterminated once a particular chain of conditions reaches a predeterminedlength (i.e., includes a pre-determined number of conditions),regardless of how many total number of respondents is associated with aparticular chain of conditions. For example, the analysis may stop oncechains of conditions of length three have been reached.

The output of the model may include a list of valid chains created bythe analysis engine 236 and a probability of receiving a particularanswer to the target question Q_T from a respondent given the set ofconditions specified by the particular chain. It will be appreciatedthat the algorithm described above is only one technique for generatinga prediction based on a model and that other types of algorithms arecontemplated as being within the scope of the present disclosure. In oneembodiment, the analysis engine 236 may generate a prediction thatincludes a percentage of respondents that has provided a particularresponse to the target question. The percentage may be calculated basedon a count of the total number of respondents that answered the targetquestion and a count of the number of respondents that provided theparticular response. However, such prediction does not provide muchinsight into the response data for the market research analyst. Inanother embodiment, the analysis engine 236 may simply calculate acorrelation measurement for each response to a valid question in thesurvey 350 in relation to a particular response to the target question.The measurements may indicate relationships between any two answers inthe survey 350.

Presentation Module

FIG. 6 illustrates a presentation module 600, in accordance with oneembodiment. The presentation module 600 is configured to generategraphical representations of a dataset that represents the surveyresponse data received from one or more respondents. In one embodiment,the presentation module 600 may be included as a separate component inthe server application 215. The presentation module 600 is configured tocreate charts to be included in one or more presentations. Thepresentations may be slide show presentations, such as a file compatiblewith Microsoft® Powerpoint, documents or white papers, such as a filecompatible with Microsoft® Word, or any other type of document ormultimedia file that may include a graphical representation of the dataset.

The presentation module 600 is capable of filtering the dataset toselect data to be represented in a chart. Queries may be applied to thesurvey response data in the survey database 242. The queries may returnthe data to be represented by the chart. The selected data may then beused to create one or more charts 650, such as line charts, bar graphs,pie charts, and the like. For example, a query may select all responsesto a particular survey question provided by a plurality of respondents.The presentation module 600 may then calculate the total number ofrespondents that submitted an answer to the question and the number ofrespondents that provided each distinct answer to the question. Thesecalculated values may then be used as data for generating the chart 650.

In one embodiment, the presentation module 600 may be configured tooutput a file or other data structure to be imported into a presentationsuch as a Microsoft® Powerpoint slideshow. The output may comprise aformatted slide including a chart as well as the underlying data used togenerate the chart. Consequently, the output includes the data showingthe individual responses to one or more questions provided byrespondents used to generate the visual representation of the data. Thefile may be imported into multiple, different presentations and sharedbetween different market research analysts. Furthermore, because theunderlying data is included in the file, the other analysts are notlimited to only viewing the graphical representation of the data in thechart, but may analyze the raw data used to generate the graphicalrepresentation as well.

Customer Portal

FIGS. 7A-7E illustrate a customer portal 700 used by a market researchanalyst to access the server application 215, in accordance with oneembodiment. The customer portal 700 includes a graphical user interface(GUI) that is displayed, either within the client application 222 or thebrowser application 224. As shown in FIG. 7A, the customer portal 700 isdisplayed within a window of the browser application 224; however,alternatively, the customer portal 700 may be displayed in a window ofthe client application 222. The GUI may include a number of UI elementsthat enable the market research analyst to interact with the serverapplication 215.

In one embodiment, the GUI includes a number of tab UI elements, such asa first tab 701, that, when selected by the market research analyst,causes an interface for one aspect of the server application 215 to bedisplayed. For example, the first tab 701 may be a survey tab, whichenables the market research analyst to create or manage surveys 350 viathe survey engine 232. The first tab 701 is associated with aninterface, displayed in a main pane of the window, that includes variousUI elements. In one embodiment, the interface includes a number ofbuttons and two text boxes. Button 711 and button 712 enable the marketresearch analyst to load a survey 350 from a file system or create a newsurvey 350, respectively. Active surveys included in the active surveylist are shown in text box 721. Button 713 and button 714 enable changesto a survey to be saved to the file system or delete a survey from theactive survey list, respectively. The market research analyst may alsoselect a survey 350 in the active survey list and edit the order andnumber of questions in the survey 350 using the text box 722. In oneembodiment, a question may be added by clicking the button 715, whichmay cause a dialog box to pop up that shows all available questions inthe survey database 242. The market research analyst may search and orfilter the questions to find one or more questions to add to theselected survey 350. The order to questions in the survey 350 may becontrolled by dragging the questions up or down in the text box 722. Themarket research analyst may also select questions in the text box 722and remove the selected questions from the survey 350 by clicking onbutton 716.

Although not shown explicitly, the interface associated with the firsttab 701 may also include other UI elements in addition to or in lieu ofthe UI elements shown in FIG. 7A. For example, the interface may includeUI elements for specifying a subset of pre-registered respondents thatshould receive the survey 350. The interface may also include a UIelement that causes a message to be sent to the subset of respondentsthat informs them that a survey 350 is available to be completed. Itwill be appreciated that the layout and content of the interfaces shownin FIGS. 7A-7E is illustrative, and that any design enabling a marketresearch analyst to access the functionality of the server application215 is contemplated as being within the scope of the present disclosure.For example, in another embodiment, the survey tab may be split intomultiple tabs, one tab for loading or deleting surveys from the activesurvey list and another tab for modifying the questions included in aparticular survey.

As shown in FIG. 7B, the GUI also includes a second tab 702, whichenables the market research analyst to create and/or edit questionsstored in the survey database 242. The interface associated with thesecond tab 702 includes a number of UI elements. A combo box 731 enablesthe market research analyst to specify a new question family by typingan identifier for the question family into the combo box 731 or select apreviously defined question family by clicking on the combo box 731 todrop down a list of all question families defined in the survey database242. A set of radio buttons 741 enables the market research analyst toselect a question type associated with the question family. A text box723 enables the market research analyst to enter an identifier for thequestion. A text box 724 enables the market research analyst to enter aprompt for the question. A text box 725 enables the market researchanalyst to enter a valid answer to the question that will be displayedto a respondent as an option for the respondent's chosen answer whencompleting the survey 350. Each valid answer may be added to a list ofvalid answers displayed in text box 726 by selecting the button 717.Valid answers may be deleted by selecting the answer in the text box 726and hitting the delete button on a keyboard. A button 718 enables themarket research analyst to load a question from the survey database 242for editing. The button 718 may cause a dialog box to open that enablesthe user to search or filter the questions in the survey database 242 toselect the particular question the market research analyst wants toedit. A button 719 enables the market research analyst to save thecurrent question to the survey database 242. A button 720 enables themarket research analyst to load a dictionary to search for a question toadd to the survey database 242. Again, the interface shown in FIG. 7B isillustrative, and other UI elements in addition to or in lieu of the UIelements described herein may be included in the interface associatedwith the second tab 702.

As shown in FIG. 7C, the GUI also includes a third tab 703, whichenables the market research analyst to create and/or edit model datastructures 450 stored in the model database 244. The interfaceassociated with the third tab 703 includes a number of UI elements. Acombo box 732 enables the market research analyst to select a particularsurvey 350 from the survey database 242. A combo box 733 enables themarket research analyst to select a particular question from theselected survey 350. The valid answers for the question may be displayedin a text box 727. The market research analyst may create a model datastructure 450 for the selected question by selecting one or more answersin the text box 727 to associate with the model data structure 450. Themarket research analyst may then specify an accuracy threshold,computation method, and stopping criteria parameters for the model datastructure 450 using the combo boxes 734, 735, and 736, respectively. Themarket research analyst may then specify a model identifier in text box728 and save the model data structure 450 to the survey database 242 byselecting the button 741. Again, the interface shown in FIG. 7C isillustrative, and other UI elements in addition to or in lieu of the UIelements described herein may be included in the interface associatedwith the third tab 703.

As shown in FIG. 7D, the GUI also includes a fourth tab 704, whichenables the market research analyst to analyze the response dataprovided by one or more respondents. The interface associated with thefourth tab 704 includes a number of UI elements. A combo box 737 enablesthe market research analyst to select a question family from the surveydatabase 242. The questions in the selected question family are thendisplayed in the text box 729. The market research analyst may thenselect a particular question in the text box 729 that the analyst wouldlike to analyze. A text box 730 may list the available analysis basesfor analyzing the raw data. The market research analyst may select oneof the analysis bases by which the data should be analyzed. Similarly, atext box 751 may list one or more filters to apply to the raw data. Thefilters, when applied to the survey response data, may only return asubset of the survey response data to be analyzed.

A button 742 may be selected to create a chart for the question based onthe analysis method chosen and the applied filters (if any). The button742 may cause a dialog box to be opened that enables the market researchanalyst to select options for the type of chart to be created. The chartmay be exported to a file system or other data structure stored in anon-volatile memory. A button 743 may export the survey response datastored in the survey database 242 to a file system or other form ofnon-volatile memory. The button 743 may cause a dialog box to be openedthat enables the market research analyst to select whether the fulldataset or only a portion of the dataset (e.g., data corresponding toone question, etc.) should be exported from the survey database 242 to afile system or other data structure stored in a non-volatile memory. Abutton 744 may cause a dialog box to be opened that enables the marketresearch analyst to create a new filter, displayed in the text box 751.A button 745 may enable weights to be applied to each question, forpurposes of creating a chart related to two or more questions. Again,the interface shown in FIG. 7D is illustrative, and other UI elements inaddition to or in lieu of the UI elements described herein may beincluded in the interface associated with the fourth tab 704.

As shown in FIG. 7E, the GUI also includes a fifth tab 705, whichenables the market research analyst to view charts created on the datatab. The interface associated with the fifth tab 704 includes a numberof UI elements. A combo box 738 enables the market research analyst toselect a specific chart to view. A second portion of the interface 761may be utilized to display the selected chart. The chart may take theform of a graphical representation of the survey response data in theform of a graph or some other format. Again, the interface shown in FIG.7E is illustrative, and other UI elements in addition to or in lieu ofthe UI elements described herein may be included in the interfaceassociated with the fifth tab 705.

Although only five tabs with different functionality have beenillustrated herein, other tabs may be included in the customer portal700. For example, a login tab may enable a market research analyst toauthenticate a session with the server application 215. Only once thesession has been authenticated may the market research analyst accessany of the other tabs. In another embodiment, a log tab may enable themarket research analyst to view records related to all activityperformed in association with the market research project. For example,the market research analyst could review a text-based log file thatindicates when respondents have completed surveys, when changes havebeen made to a survey, or when questions have been added to the surveydatabase 242.

FIG. 8A illustrates a flow chart of a method 800 for generating a surveybased on one or more models, in accordance with one embodiment. Themethod 800 may be performed by the server application 215 in response toreceiving a completed survey. At step 802, response data for a firstsurvey 350 is received from a first respondent. The response data maycontain the respondent's answers to the one or more questions includedin the survey 350 and may be stored in the survey database 242. Thesurvey engine 232 may transmit a message to the analysis engine 236 thata survey 350 has been completed by the first respondent. At step 804, inresponse to receiving the message, the analysis engine 236 selects amodel data structure 450 from an active model list associated with thesurvey 350. Again, each model data structure 450 is associated with acorresponding model, and the active model list indicates which model(s)should be run by the analysis engine 236 after a particular survey 350has been completed. The analysis engine 236 will run each modelassociated with a model data structure 450 in the active model list.

At step 806, the analysis engine 236 runs the selected model to generatea prediction. The model may be updated based on the response datareceived from the first respondent prior to running the model. In oneembodiment, the prediction comprises the result of one or morecomputations performed by implementing an algorithm specified by thecomputation method parameter of a corresponding model data structure450. In one embodiment, the result of the computation(s) may be togenerate a prediction associated with a question, a measurement of theaccuracy of such prediction, and information that defines relationshipsbetween the question and one or more additional questions. Themeasurement of the accuracy may be compared against the accuracythreshold parameter included in the corresponding model data structure450. If the measurement of the accuracy meets or exceeds the accuracythreshold parameter, then one or more actions specified in the stoppingcriteria parameter of the corresponding model data structure 450 may beperformed. At step 808, the analysis engine 236 determines if anothermodel needs to be run. If the active model list contains at least onemore model data structure 450 associated with a model that has not beenrun after the response data was received from the first respondent, thenthere is at least one additional model that needs to be run and themethod 800 returns to step 804 where another model data structure 450 isselected from the active model list.

Once all the models have been run, at step 810, the survey engine 232may generate a second survey 350 based on the output of the model(s).The second survey 350 may not include at least one question included inthe first survey 350 and may include one or more additional questionsnot included in the first survey 350. At least one question included inthe first survey 350 and not included in the second survey 350 may beassociated with a model data structure 450 having an accuracy thresholdparameter that was met or exceeded by a corresponding measurement of theaccuracy generated by the analysis engine 236. In other words, thestopping criteria of the model data structure 450, when triggered by thecomparison of the measured accuracy versus the accuracy thresholdparameter, may cause at least one question to be removed from the firstsurvey 350 to generate the second survey 350. The method terminates atstep 810 and the second survey 350 may be transmitted to one or moreadditional respondents to be completed to generate new response data.

FIG. 8B illustrates a flow chart of a method 850 for dynamicallygenerating a survey, in accordance with one embodiment. In oneembodiment, questions in a survey 350 are generated dynamically, basedon a model that has been updated based on information received from oneor more respondents. The model may cause specific questions to beremoved from a dynamically generated survey 350 based on answersreceived from the respondent to earlier questions in the survey 350. Inother words, each of the questions in the survey 350 and the order ofthe questions may be selected dynamically as the respondent completesthe survey 350, based on the output of the model that has beencontinuously updated using the responses provided by the respondent.

At step 852, the survey engine 232 causes a first question to bedisplayed to a respondent. The first question may include one or moretarget responses that can be selected by the respondent. At step 854,the survey engine 232 receives a response to the first question from therespondent. At step 856, the survey engine 232 determines whether theresponse matches a first target response or a second target response. Inone embodiment, the survey engine 232 includes logic for identifyingwhich target response associated with the question has been selected asan answer by the respondent. In another embodiment, the survey engine232 transmits a message to the analysis engine 236 that indicates aresponse has been received. The analysis engine 236 may then update andrun a model based on the received response. The output of the model fromthe analysis engine 236 may indicate whether the first target responsewas received or the second target response was received and may be usedto determine whether a second question or a third question should bedisplayed to the respondent.

If the first target response was received, then, at step 858, the surveyengine 232 causes the second question to be displayed to the respondent.However, if the second target response was received, then, at step 860,the survey engine 232 causes the third question to be displayed to therespondent. It will be appreciated that the method 850 may be repeated anumber of times for a given survey 350 as the respondent answers eachquestion included in the survey 350. Consequently, the model is updatedin real-time after each provided answer and the contents of the survey350 are a reflection of the output of the model. In one embodiment,real-time updates may include providing a real-time update to questionspresented based on use of a model. In a separate embodiment, real-timeupdates may include providing a real-time update to a model which isthen used to update questions based on the updated model.

In one embodiment, when the questions for a survey are generateddynamically, the market analyst may specify limitations associated withthe survey. The limitations may include a set number of questions, a setamount of time for the respondent to answer questions, etc. Otherlimitations may be dependent on other variables, such as dependent onthe responses provided by the respondent or dependent on whether anyrelationships between questions are identified by the model(s). In suchembodiments, the method 850 may be repeated until one of the limitationsis met. For example, questions may be generated dynamically until therespondent has answered 50 questions. Once the limitation is met, thesurvey 350 is completed and method 850 terminates.

FIG. 9 illustrates an exemplary system 900 in which the variousarchitecture and/or functionality of the various previous embodimentsmay be implemented. As shown, a system 900 is provided including atleast one central processor 901 that is connected to a communication bus902. The communication bus 902 may be implemented using any suitableprotocol, such as PCI (Peripheral Component Interconnect), PCI-Express,AGP (Accelerated Graphics Port), HyperTransport, or any other bus orpoint-to-point communication protocol(s). The system 900 also includes amain memory 904. Control logic (software) and data are stored in themain memory 904 which may take the form of random access memory (RAM).

The system 900 also includes input devices 912, a graphics processor906, and a display 908, i.e. a conventional CRT (cathode ray tube), LCD(liquid crystal display), LED (light emitting diode), plasma display orthe like. User input may be received from the input devices 912, e.g.,keyboard, mouse, touchpad, microphone, and the like. In one embodiment,the graphics processor 906 may include a plurality of shader modules, arasterization module, etc. Each of the foregoing modules may even besituated on a single semiconductor platform to form a graphicsprocessing unit (GPU).

In the present description, a single semiconductor platform may refer toa sole unitary semiconductor-based integrated circuit or chip. It shouldbe noted that the term single semiconductor platform may also refer tomulti-chip modules with increased connectivity which simulate on-chipoperation, and make substantial improvements over utilizing aconventional central processing unit (CPU) and bus implementation. Ofcourse, the various modules may also be situated separately or invarious combinations of semiconductor platforms per the desires of theuser.

The system 900 may also include a secondary storage 910. The secondarystorage 910 includes, for example, a hard disk drive and/or a removablestorage drive, representing a floppy disk drive, a magnetic tape drive,a compact disk drive, digital versatile disk (DVD) drive, recordingdevice, universal serial bus (USB) flash memory. The removable storagedrive reads from and/or writes to a removable storage unit in awell-known manner.

Computer programs, or computer control logic algorithms, may be storedin the main memory 904 and/or the secondary storage 910. Such computerprograms, when executed, enable the system 900 to perform variousfunctions. The memory 904, the storage 910, and/or any other storage arepossible examples of computer-readable media.

In one embodiment, the architecture and/or functionality of the variousprevious figures may be implemented in the context of the centralprocessor 901, the graphics processor 906, an integrated circuit (notshown) that is capable of at least a portion of the capabilities of boththe central processor 901 and the graphics processor 906, a chipset(i.e., a group of integrated circuits designed to work and sold as aunit for performing related functions, etc.), and/or any otherintegrated circuit for that matter.

Still yet, the architecture and/or functionality of the various previousfigures may be implemented in the context of a general computer system,a circuit board system, a game console system dedicated forentertainment purposes, an application-specific system, and/or any otherdesired system. For example, the system 900 may take the form of adesktop computer, laptop computer, server, workstation, game consoles,embedded system, and/or any other type of logic. Still yet, the system900 may take the form of various other devices including, but notlimited to a personal digital assistant (PDA) device, a mobile phonedevice, a television, etc.

Further, while not shown, the system 900 may be coupled to a network(e.g., a telecommunications network, local area network (LAN), wirelessnetwork, wide area network (WAN) such as the Internet, peer-to-peernetwork, cable network, or the like) for communication purposes.

While various embodiments have been described above, it should beunderstood that they have been presented by way of example only, and notlimitation. Thus, the breadth and scope of a preferred embodiment shouldnot be limited by any of the above-described exemplary embodiments, butshould be defined only in accordance with the following claims and theirequivalents.

What is claimed is:
 1. A system comprising: a server computer executinga server application to: receive first information from one or morerespondents, using an analysis engine of the server computer, whereinthe first information includes at least one response to a first questionincluded in a first survey, wherein the first information includesaggregated data; update a model including the first survey based on thefirst information, wherein the updating includes: performing acomputation based on the first information to determine an accuracyrelated to a prediction for the first question, wherein the accuracyincludes an estimation of whether an expected answer submitted by one ofthe one or respondents would result in a statistically relevantdatapoint, wherein the prediction is generated by calculating arelationship between the first question and each of: an answer to asecond question by a same respondent of the one or more respondents inthe first survey, an answer to the second question by a differentrespondent of the one or more respondents in the first survey, an answerto the second question by a different respondent of the one or morerespondents in a different survey, and other data that is unrelated tothe first or second questions, including at least one of purchase dataand website usage, determining that the accuracy is above a level of anaccuracy threshold included in the model, and in response to determiningthat the accuracy is above the level of the accuracy threshold includedin the model, performing one or more actions specified in a stoppingcriteria of the model; generate a second survey, using a survey engineof the server computer, based on the updated model, by: determiningwhether the stopping criteria is satisfied, and when the stoppingcriteria is satisfied, then the one or more actions specified in thestopping criteria cause at least one question to be removed from thefirst survey to generate the second survey, and when the stoppingcriteria is not satisfied, then the one or more actions specified in thestopping criteria cause at least one additional question to be added tothe second survey; receiving second information from one or moreadditional respondents; running the updated model to determine aprobability of a first respondent of the one or more additionalrespondents to provide an answer to a third question associated with afirst aspect of the second information; determining whether the thirdquestion is used to predict a response to a fourth question; based onthe probability, determining whether to remove or add the third questionto the second survey, wherein if the third question is predicted byother questions, then the third question is removed from the secondsurvey unless the third question is used to predict the response to thefourth question; updating the model including the second survey based onthe second information to generate a third survey, using the surveyengine.
 2. The system of claim 1, wherein the server applicationincludes the survey engine and the analysis engine, the survey engineconfigured to: receive the first information from the one or morerespondents; and in response to receiving the first information,transmit a message to the analysis engine to update the model.
 3. Thesystem of claim 2, wherein the analysis engine is configured to transmita message to the survey engine in response to updating the model, andthe survey engine is configured to generate the second survey inresponse to receiving the message.
 4. The system of claim 1, wherein themodel comprises a data structure storing a number of parameters, theparameters including a survey identifier, a question identifier, atarget response, an accuracy threshold, a computation method, andstopping criteria.
 5. The system of claim 1, further comprising: aclient application included on a client computer, the client applicationconfigured to communicate with the server application via a network. 6.The system of claim 5, wherein the client application comprises a thinclient implemented within a browser, the client application configuredto communicate with the server application via HTTP request and HTTPresponse messages.
 7. The system of claim 5, wherein the clientapplication is configured to display one or more charts, each chart is agraphical representation of the first or second information.
 8. Thesystem of claim 1, wherein generating the second survey comprises:causing the third question to be displayed to a respondent; receiving aresponse to the third question from the respondent; determining whetherthe response matches a first target response or a second targetresponse; and if the response matches the first target response, causingthe fourth question to be displayed to the respondent, or if theresponse matches the second target response, causing a fifth question tobe displayed to the respondent.
 9. The system of claim 1, whereinperforming the computation comprises implementing an algorithm specifiedby the model to determine the accuracy.
 10. The system of claim 1,wherein the first information comprises a number of responses to acorresponding number of questions in the first survey for each of theone or more respondents.
 11. The system of claim 1, wherein thecomputation, using a computation engine of the server computer, includesrunning a plurality of models for the model.
 12. The system of claim 1,wherein the computation is used to generate the prediction, ameasurement of the accuracy, and secondary information that defines oneor more relationships between the first question and one or moreadditional questions.
 13. The system of claim 1, wherein the one or moreactions specified by the stopping criteria include deactivating thefirst survey, activating a new survey, or removing one or more questionsfrom the first survey.
 14. A non-transitory computer-readable storagemedium storing instructions that, when executed by a processor, causethe processor to perform steps comprising: receiving, using theprocessor, first information from one or more respondents, using ananalysis engine of a server computer, wherein the first informationincludes at least one response to a first question included in a firstsurvey, wherein the first information includes aggregated data;updating, using the processor, a model including the first survey basedon the first information, wherein the updating includes: performing acomputation based on the first information to determine an accuracyrelated to a prediction for the first question, wherein the accuracyincludes an estimation of whether an expected answer submitted by one ofthe one or respondents would result in a statistically relevantdatapoint, wherein the prediction is generated by calculating arelationship between the first question and each of: an answer to asecond question by a same respondent of the one or more respondents inthe first survey, an answer to the second question by a differentrespondent of the one or more respondents in the first survey, an answerto the second question by a different respondent of the one or morerespondents in a different survey, and other data that is unrelated tothe first or second questions, including at least one of purchase dataand website usage, determining that the accuracy is above a level of anaccuracy threshold included in the model, and in response to determiningthat the accuracy is above the level of the accuracy threshold includedin the model, performing one or more actions specified in a stoppingcriteria of the model; generating, using the processor, a second survey,using a survey engine of the server computer, based on the updatedmodel, by: determining whether the stopping criteria is satisfied, andwhen the stopping criteria is satisfied, then the one or more actionsspecified in the stopping criteria cause at least one question to beremoved from the first survey to generate the second survey, and whenthe stopping criteria is not satisfied, then the one or more actionsspecified in the stopping criteria cause at least one additionalquestion to be added to the second survey; receiving second informationfrom one or more additional respondents; running the updated model todetermine a probability of a first respondent of the one or moreadditional respondents to provide an answer to a third questionassociated with a first aspect of the second information; determiningwhether the third question is used to predict a response to a fourthquestion; based on the probability, determining whether to remove or addthe third question to the second survey, wherein if the third questionis predicted by other questions, then the third question is removed fromthe second survey unless the third question is used to predict theresponse to the fourth question; updating the model, using theprocessor, including the second survey based on the second informationto generate a third survey, using the survey engine.
 15. Thenon-transitory computer-readable storage medium of claim 14, wherein themodel comprises a data structure storing a number of parameters, theparameters including a survey identifier, a question identifier, atarget response, an accuracy threshold, a computation method, andstopping criteria.
 16. The non-transitory computer-readable storagemedium of claim 14, wherein generating the second survey comprises:causing the third question to be displayed to a respondent; receiving aresponse to the third question from the respondent; determining whetherthe response matches a first target response or a second targetresponse; and if the response matches the first target response, causingthe fourth question to be displayed to the respondent, or if theresponse matches the second target response, causing a fifth question tobe displayed to the respondent.
 17. A method, comprising: receiving, bya server computer, first information from one or more respondents, usingan analysis engine of a server computer, wherein the first informationincludes at least one response to a first question included in a firstsurvey, wherein the first information includes aggregated data; updatinga model including the first survey based on the first information,wherein the updating includes: performing a computation based on thefirst information to determine an accuracy related to a prediction forthe first question, wherein the accuracy includes an estimation ofwhether an expected answer submitted by one of the one or respondentswould result in a statistically relevant datapoint, wherein theprediction is generated by calculating a relationship between the firstquestion and each of: an answer to a second question by a samerespondent of the one or more respondents in the first survey, an answerto the second question by a different respondent of the one or morerespondents in the first survey, an answer to the second question by adifferent respondent of the one or more respondents in a differentsurvey, and other data that is unrelated to the first or secondquestions, including at least one of purchase data and website usage,determining that the accuracy is above a level of an accuracy thresholdincluded in the model, and in response to determining that the accuracyis above the level of the accuracy threshold included in the model,performing one or more actions specified in a stopping criteria of themodel; generating a second survey, using a survey engine of the servercomputer, based on the updated model, by: determining whether thestopping criteria is satisfied, and when the stopping criteria issatisfied, then the one or more actions specified in the stoppingcriteria cause at least one question to be removed from the first surveyto generate the second survey, and when the stopping criteria is notsatisfied, then the one or more actions specified in the stoppingcriteria cause at least one additional question to be added to thesecond survey; receiving second information from one or more additionalrespondents; running the updated model to determine a probability of afirst respondent of the one or more additional respondents to provide ananswer to a third question associated with a first aspect of the secondinformation; determining whether the third question is used to predict aresponse to a fourth question; based on the probability, determiningwhether to remove or add the third question to the second survey,wherein if the third question is predicted by other questions, then thethird question is removed from the second survey unless the thirdquestion is used to predict the response to the fourth question;updating the model including the second survey based on the secondinformation to generate a third survey, using the survey engine.
 18. Themethod of claim 17, wherein the model comprises a data structure storinga number of parameters, the parameters including a survey identifier, aquestion identifier, a target response, an accuracy threshold, acomputation method, and stopping criteria.
 19. The method of claim 17,wherein generating the second survey comprises: causing the thirdquestion to be displayed to a respondent; receiving a response to thethird question from the respondent; determining whether the responsematches a first target response or a second target response; and if theresponse matches the first target response, causing the fourth questionto be displayed to the respondent, or if the response matches the secondtarget response, causing a fifth question to be displayed to therespondent.